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(54) MPEG coded picture decoding apparatus 

(57) An MPEG coded picture decoding apparatus 
includes a stream buffer memory (1) for storing an 
MPEG stream, a pack header processing section (3) for 
separating a pack header of the MPEG stream and 
extracting SCR information, a PES packet processing 
section (4) for separating a header of a PES stream, 
extracting PTS information and separating packets, a 
GOP detection section (5) for detecting a GOP of a 
video stream, a video buffer memory (6) for storing the 
video stream, a video MPEG decoding section (8) for 

FIG. 1 



MPEG decoding the video stream, a decoded data stor- 
age section (9) for storing and managing one picture 
frame, a video display processing section (10) for stor- 
ing several picture frames and reading out at a display 
timing, and an error information processing section (14) 
for supervising a picture frame memory at each display 
timing and developing a video re-sending request when 
the picture frame memory is in an irregular condition. 
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Description 

This invention relates to an MPEG (Moving Picture 
Experts Group) coded picture decoding apparatus, and 
more particularly to an MPEG coded picture decoding s 
apparatus which includes an error picture re-sending 
section which is used when MPEG coded picture data 
suffer from a data error such as a miss of data in a 
transmission line or the like. 

In a conventional MPEG coded picture decoding 10 
apparatus, when an uncorrectable code error, for exam- 
ple, a transmission error such as a long burst error, 
occurs with MPEG coded picture data in a transmission 
line, a decoded picture suffers from significant degrada- 
tion in picture quality or a missing portion is produced in is 
a picture. Therefore, the degraded picture or the picture 
having the missing part is replaced with another picture 
in the past on the decoding side to correct the picture 
smoothly with respect to a surrounding picture. 

For example, in a moving picture decoding appara- 20 
tus disclosed in Japanese Patent Latd-Open Application 
No. Heisei 5-153574, a block with which an uncorrecta- 
ble code error has occurred is replaced with a block pic- 
ture produced based on motion vectors in the past 
determined based on frame pictures in the past and 25 
then is decoded thereby to correct a missing portion of 
the picture, which arises from the code error, smoothly 
with respect to a surrounding picture irrespective of 
whether or not the decoded picture exhibits some 
motion. 30 

In the conventional MPEG coded picture decoding 
apparatus described above, since a degraded or miss- 
ing portion of a decoded picture caused by a transmis- 
sion error is merely replaced with a block picture 
produced based on motion vectors in the past, blurring 35 
of the contour of the corrected image cannot be 
avoided, and it is impossible to decode and display a 
picture same as the picture which does not suffer from 
the degradation or miss. 

It is an object of the present invention to provide an 40 
MPEG coded picture decoding apparatus which can 
decode and display, even if a miss of MPEG coded 
video data occurs, for example, on the MPEG coding 
multiplexing side, in a transmission line or the like, video 
data quite same as original data, which do not suffer 45 
from the miss, in units of one picture frame or one GOP 
(Group of Pictures) and can display pictures for several 
picture frames successively without suffering from a 
miss of a picture caused by the miss of the data. 

In order to attain the object described above, so 
according to the present invention, there is provided an 
MPEG coded picture decoding apparatus for MPEG 
coded picture data transmitted thereto from an MPEG 
coding multiplexing side by means of a decoding ele- 
ment, comprising a memory for temporarily storing ss 
video data MPEG decoded by the decoding element in 
units of one picture frame, a memory management sec- 
tion for supervising the memory to detect whether or not 
picture data for one frame are stored into the memory 



completely without a miss, a video display processing 
section including a display memory into which video 
data for one picture frame successively read out from 
the memory are temporarily stored for several picture 
frames and from which the video data are read out so as 
to be displayed in the same order in units of one picture 
frame, an error processing section for checking memory 
storage information at an output of the memory man- 
agement section at each video display timing and for 
outputting, when the memory storage information is not 
outputted from the memory management section, error 
information determining that a miss of data has 
occurred with a picture frame stored immediately before 
the display timing then and delivering a re-sending 
request to the MPEG coding multiplexing side to re- 
send video data for the one picture frame, with which 
the miss of data has occurred, before the video data of 
the picture frame, with which the miss of data has 
occurred, or a GOP including the picture frame are read 
out from the video display processing section, and a 
control section for storing video data of the one picture 
frame or the GOP resent from the MPEG coding multi- 
plexing side into the display memory of the video display 
processing section by a time at which the video data are 
to be displayed. 

In the MPEG coded picture decoding apparatus, 
MPEG coded picture data are temporarily stored into 
the memory in units of one picture frame, and it is 
detected by supervision by the memory management 
section whether or not picture data for one frame are 
stored into the memory completely without a miss. 
Then, video data for one picture frame successively 
read out from the memory are temporarily stored for 
several picture frames into the display memory and then 
successively read out from the display memory in the 
same order in units of one picture frame so as to be dis- 
played. Meanwhile, the storage condition of the memory 
for one picture frame is checked at each video display 
timing, and when picture data for one picture frame are 
not stored completely, it is determined that a miss of 
data has occurred with a picture frame stored immedi- 
ately before the display timing then. Then, a re-sending 
request is delivered to the MPEG coding multiplexing 
side to re-send video data for the one picture frame, with 
which the miss of data has occurred, before the video 
data of the picture frame, with which the miss of data 
has occurred, or a GOP including the picture frame are 
read out from the video display processing section. 
Then, video data re-sent from the MPEG coding multi- 
plexing side are stored newly into the display memory 
by a time at which the video data are to be displayed. 
Consequently, even if a miss of MPEG coded data 
occurs in the MPEG coding multiplexing side, the trans- 
mission line or the like, video data same as the original 
data which do not suffer from any miss can be displayed 
in units of a one picture frame or in units of a GOP. 
Accordingly successive picture frames including the 
picture frame with which the miss of data has occurred 
can be displayed successively without a miss of any pic- 
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ture. 

Further, since the error information can be used as 
a requesting signal to stop sending out of video data 
from the MPEG coding side following the picture frame 
with which the miss of data has occurred, even if the 5 
MPEG coded data after the miss of data are influenced 
by noise of the transmission line or by some other cause 
and suffer from an error, the display video data of a 
wrong picture frame or frames are not outputted. Con- 
sequently, a disagreeable feeling which may arise from io 
such wrong picture frame or frames is not given to the 
video viewer at all. 

According to another aspect of the present inven- 
tion to provide an MPEG coded picture decoding appa- 
ratus, comprising a first memory for successively and rs 
temporarily storing a multiplexed stream inputted 
thereto from an MPEG coding multiplexing side via a 
transmission line and a line input interface and multi- 
plexed from MPEG coded bit streams of a plurality of 
media including video and audio data, a pack header 20 
processing section for separating a pack header of the 
multiplexed stream read out from the first memory in 
response to a first control signal and extracting system 
clock reference information for a system clock refer- 
ence, a packet processing section for separating a 25 
packet header from the multiplexed stream from the 
packet header separation section from which the pack 
header has been separated, extracting presentation 
time stamp information for a time reference and sepa- 
rating the multiplexed stream from the pack header 30 
processing section in units of a packet of each of the 
plurality of media, a GOP detection section for detecting 
a GOP from an MPEG video stream of video packets 
separated by the packet processing section, a first con- 
trol section for outputting the first control signal and a 35 
second control signal in response to a GOP detection 
signal from the GOP detection section, a second mem- 
ory for temporarily storing the MPEG video stream of 
the video packets separated by the packet processing 
section and being controlled by the second control sig- 40 
nal from the first control section to read out the MPEG 
video stream temporarily stored therein in units of a 
GOP, a video MPEG decoding section for being control- 
led by a third control signal to MPEG decode video data 
in units of one picture frame outputted from the second 45 
memory, a third memory for temporarily storing 
decoded video data for one picture frame from the video 
MPEG decoding section, a memory management sec- 
tion for supervising a storage condition of the decoded 
video data for one picture frame into a storage region of so 
the third memory and controlling, when the storage con- 
dition is regular, the third memory to output the decoded 
video data for one picture frame stored in the third mem- 
ory, but outputting, when the storage condition is irregu- 
lar, first error information, a fourth memory for 55 
temporarily storing the decoded video data for one pic- 
ture frame outputted from the third memory for a first 
number of picture frames, a video display control sec- 
tion for controlling the decoded video data read out in 



units of one picture frame from the fourth memory in 
response to a fourth control signal so that the decoded 
video data may be outputted at a video display timing, a 
digital to analog conversion section for performing dig- 
ital to analog conversion of the coded video data for 
each one picture frame from the video display control 
section and outputting and supplying the resulting ana- 
log decoded video data as an MPEG decoded picture 
signal to an external video viewer, a second control sec- 
tion for outputting the third and fourth control signals in 
response to a fifth control signal so that the third control 
signal is supplied to the second memory and the fourth 
control signal is supplied to the fourth memory, a display 
timing production section for producing and supplying 
the video display timing to the video display control sec- 
tion in order to allow the MPEG decoded picture signal 
outputted from the digital to analog conversion section 
to be displayed at a timing on a picture monitor of the 
video viewer, an error information processing section for 
supervising an input thereto of the first error information 
outputted from the memory management section at a 
timing of a period equal to that of the video display tim- 
ing outputted from the display timing production section 
and outputting, when the first error information is input- 
ted thereto, a video re-sending request signal based on 
the error information to the MPEG coding multiplexing 
side to request for re-sending of video data of the same 
picture frame as that of the video data with regard to 
which the first error information is outputted, a clock 
generation section for generating a decoding side clock 
signal to be used for MPEG decoding based on the sys- 
tem clock reference information extracted by the pack 
header processing section, a time information produc- 
tion section for producing decoding side time informa- 
tion to be used for MPEG decoding based on the 
presentation time stamp information extracted by the 
packet processing section, a synchronization timing 
production section for producing an AV synchronization 
timing for synchronization between the video and audio 
data upon MPEG decoding based on the decoding side 
time information outputted from the time information 
production section, and a CPU control section for 
receiving the decoding side clock signal from the clock 
generation section, the decoding side time information 
from the time information production section, the GOP 
detection signal from the GOP detection section and the 
video re-sending request signal from the error informa- 
tion processing section to drive a host CPU to control 
the MPEG decoding processing of the MPEG decoding 
section via a CPU bus and to output the fifth control sig- 
nal to control the second control section and sending 
out the video re-sending request signal to the MPEG 
coding multiplexing side via a line output interface and 
the transmission line. 

The above and other objects, features and advan- 
tages of the present invention will become apparent 
from the following description and the appended claims, 
taken in conjunction with the accompanying drawings in 
which like parts or elements are denoted by like refer- 
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ence characters. 

FIG. 1 is a block diagram of an MPEG coded picture 
decoding apparatus showing a preferred embodi- 
ment of the present invention; s 
FIG. 2 is a diagrammatic view illustrating an exam- 
ple of multiplexing by the MPEG coded • picture 
decoding apparatus of FIG. 1 when multimedia on- 
demand information is MPEG multiplexed; 
FIG. 3 is a diagrammatic view illustrating an exam- w 
pie of an MPEG coding sequence and an inter-pic- 
ture prediction method within one GOP by the 
MPEG coded picture decoding apparatus of FIG. 1 ; 
and * 
FIG. 4 is a diagrammatic view showing an example 15 
of a stream data array of an MPEG coded bit 
stream of the MPEG coded picture decoding appa- 
ratus of FIG. 1 . 

Referring first to FIG. 1 , there is shown an MPEG 20 
coded picture decoding apparatus to which the present 
invention is applied. The MPEG coded picture decoding 
apparatus shown includes a stream buffer memory 1 , a 
buffer memory control section 2, a pack header 
processing section 3, a PES packet processing section 25 
4, a GOP detection section 5, a video buffer memory 6, 
a buffer memory control section 7, a video MPEG 
decoding section 8, a decoded data storage section 9, a 
video display processing section 10, a display timing 
production section 1 1 , a video D/A conversion section 30 
12, a frame check detection section 13, an error infor- 
mation processing section 14, a clock generation sec- 
tion 15, a time information production section 16, an 
audio buffer memory 19, an AV synchronisation timing 
production section 20, an audio MPEG decoding sec- 35 
tion 21 , an audio D/A conversion section 22, and a CPU 
control section 25. 

The stream buffer memory 1 temporarily stores a 
bit stream of MPEG coded multiplexed data sent out 
from a server side, that is, an MPEG coding multiplexing 40 
side via a high speed digital data private line, which 
flows digital data at a high speed in a fixed rate, and 
inputted thereto from a line interface section 26. The 
MPEG coded multiplex data are composed of multi- 
plexed bit streams of a plurality of media of multimedia 45 
on-demand information such as moving pictures and 
sound. 

The pack header processing section 3 includes a 
header detection and separation section 30 which sep- 
arates a pack header of a bit stream of a pack layer read so 
out from the stream buffer memory 1 under the control 
of the buffer memory control section 2 and detects a 
stream identification code (stream ID) of the separated 
pack header, and an SCR extraction section 31 for 
extracting, based on the stream identification code ss 
detected by the header detection and separation sec- 
tion 30. SCR code information for a system clock refer- 
ence disposed subsequently to the stream identification 
code. 



The PES packet processing section 4 includes a 
header separation section 40 for separating a PES 
(Packetized Elementary Stream) packet header from a 
stream for each medium factor packetized from a bit 
stream of an output of the pack header processing sec- 
tion 3 from which packet headers have been separated, 
and a PTS extraction section 41 for detecting a stream 
ID in the PES packet header separated by the header 
separation section 40 and extracting PTS (Presentation 
Time Stamp) code information which is video display 
time information in an optional PES header not shown 
disposed subsequently to the stream ID. The PES 
packet processing section 4 further includes a packet 
separation section 42 for separating, based on the 
stream ID in the PES packet header, video packets and 
audio packets from each other for each pack from the bit 
stream, from which PES packet headers have been 
separated, and outputting the video packets and the 
audio packets to a video packet output port, a GOP 
(Group Of Pictures) output port and an audio packet 
output port. 

The PES packet processing section 4 detects a 
GOP flag which indicates the start of a VOP video 
stream from video packets for each of separated pack- 
ets inputted thereto from the GOP output port of the 
PES packet processing section 4 from which PES 
packet headers have been separated. 

The video buffer memory 6 temporarily stores a 
video stream inputted thereto from the video packet out- 
put port of the PES packet processing section 4. From 
the video buffer memory 6, picture data in the stored bit 
stream are read out in units of a GOP under the control 
of the buffer memory control section 2 and read out in 
units of one picture frame under the control of the buffer 
memory control section 7. 

The video MPEG decoding section 8 MPEG 
decodes video data read out for each one picture frame 
of a unit of a GOP from the video buffer memory 6 under 
the control of a host CPU 18 via a CPU bus 17 and a 
host memory 23. 

The decoded data storage section 9 includes a pic- 
ture decoded data frame memory 90 for temporarily 
storing video data for one picture frame decoded by the 
video MPEG decoding section 8, and a memory man- 
agement section 91 for supervising the storage condi- 
tion of video data in a memory region temporarily stored 
in the picture decoded data frame memory 90 and stop- 
ping, when a data error such as a miss of video data in 
a predetermined memory region is detected, outputting 
from the picture decoded data frame memory 90 of 
video data of the one picture frame with which the data 
error has occurred and outputting an error detection sig- 
nal, but outputting, if the video data are stored into the 
predetermined memory region without a miss of them, a 
regular storage signal simultaneously upon completion 
of the storage. 

The video display processing section 10 includes a 
video display frame memory 100 for temporarily storing 
miss-free regular video data for several picture frames 
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successively inputted thereto from the decoded data 
storage section 9. and a display control section 101 for 
being controlled by a video display timing signal output- 
ted from the display timing production section 1 1 to out- 
put video data for one picture frame read out from the 5 
video display frame memory 100 under the control of 
the buffer memory control section 7 so that the video 
data may be displayed on a picture monitor of an exter- 
nal viewer not shown. 

The video D/A conversion section 12 performs dig- 10 
ital to analog (D/A) conversion of display video data out- 
putted from the video display processing section 10 and 
outputs resulting analog data to the picture monitor of 
the external viewer mentioned above. * 

The video display processing section 10 receives, is 
as input signals thereto, a regular storage signal output- 
ted from the memory management section 91 of the 
decoded data storage section 9 and a video display tim- 
ing signal outputted from the display timing production 
section 1 1 . Then, the video display processing section 20 
10 determines, if a regular storage signal is not inputted 
thereto when a video display timing signal is inputted 
thereto, that video data stored in the picture decoded 
data frame memory 90 suffer from a data error such as 
a miss, and outputs an error detection signal together 25 
with the error information. 

The error information processing section 14 out- 
puts, when a data error detection signal is inputted 
thereto from the frame check detection section 13, 
based on error information then, a video re-sending 30 
request signal for requesting for re-sending of video 
data of a picture frame, which which the data error has 
occurred, from the MPEG coding multiplexing side, a 
video stopping request signal for requesting for stopping 
of sending out of video data from the MPEG coding mul- 35 
tiplexing side or a like signal. 

The clock generation section 15 produces an intra- 
apparatus clock signal for MPEG decoding processing 
and so forth based on SCR code information extracted 
by the SCR extraction section 31 of the pack header 40 
processing section 3. 

The time information production section 16 pro- 
duces intra-apparatus time information for a read timing 
and an audio/video synchronization (AV synchroniza- 
tion) timing of MPEG decoded picture frame data based 45 
on PTS code information extracted by the PTS extrac- 
tion section 41 of the PES packet processing section 4. 

The display timing production section 1 1 outputs a 
display timing for picture frame picture display and a 
checking timing for checking of a picture frame memory so 
storage condition and supplies them to the display con- 
trol section 101 of the video display processing section 
10 and the frame check detection section 13. 

The buffer memory control section 2 controls read- 
ing out of pack stream data temporarily stored in the 55 
stream buffer memory 1 in response to a GOP detection 
signal supplied thereto from the GOP detection section 
5 after each period of a GOP flag and controls, in 
response to the same GOP detection signal, reading 



out of 1 video PES stream data temporarily stored in the 
video buffer memory 6 and disposed subsequently to 
the GOP flag. 

The buffer memory control section 7 outputs a con- 
trol signal for reading out video data for one picture 
frame to be MPEG decoded in a picture frame period 
from the video buffer memory 6 and outputs another 
control signal for reading out video data of a predeter- 
mined number of picture frames temporarily stored in 
the video display frame memory 1 00 of the video dis- 
play processing section 10 one by one picture frame in 
the same picture frame period under the control of the 
CPU control section 25. 

The audio buffer memory 19 temporarily stores an 
audio stream inputted thereto from the audio packet 
output port of the PES packet processing section 4. 

The AV synchronization timing production section 
20 produces an AV synchronization timing signal for 
synchronization of video and audio data of MPEG 
decoded data in response to intra-apparatus time infor- 
mation outputted from the time information production 
section 16, and supplies the AV synchronisation timing 
signal to the video MPEG decoding section 8 and the 
audio MPEG decoding section 21 . 

The audio MPEG decoding section 21 MPEG 
decodes an audio stream read out from the audio buffer 
memory 19 under the control of the host CPU 18 via the 
CPU bus 1 7 and the host memory 23. 

The audio D/A conversion section 22 performs dig- 
ital to analog conversion of decoded audio data output- 
ted from the audio MPEG decoding section 21 and 
sends out the resulting analog data to the video viewer. 

The CPU control section 25 receives an intra-appa- 
ratus clock signal from the clock generation section 15. 
intra-apparatus time information from the time informa- 
tion production section 16, a GOP detection signal from 
the GOP detection section 5 and an error information 
signal from the error information processing section 14 
and controls the host CPU 18 via the CPU bus 17 and 
the host memory 23 to effect control and transfer of data 
to and from the video MPEG decoding section 8, the 
audio MPEG decoding section 21, the buffer memory 
control section 7 and so forth. Further, the CPU control 
section 25 controls sending out of error information 
such as a request signal for requesting for re-sending of 
video data of a picture frame, with which a data error 
such as a miss of data has occurred, or for stopping of 
sending out of video data to the MPEG coding multi- 
plexing side via a line interface section 24. 

FIG. 2 illustrates an example of multiplexing when, 
for example, a moving picture signal, an audio signal 
and other signals (for example, a character data signal) 
of multimedia on-demand information are multiplexed. 
FIG. 3 illustrates an example of a coding order in one 
GOP of an I picture, P pictures and B pictures where 
one GOP includes 15 frames and an inter-picture pre- 
diction method between pictures. FIG. 4 illustrates an 
example of a stream data array of an MPEG coded bit 
stream. 
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Referring to FIG. 1 to 4, when a plurality of MPEG 
coded bit streams of moving picture data, audio data, 
character data and some other data of multimedia on- 
demand information are to be transmitted to a viewer 
side via a transmission line, data of the bit streams 5 
within substantially same times are collected to form a 
packet, and a plurality of such packets are individually 
time division multiplexed to form a plurality of packs. 

FIG. 2 illustrates an example of data structure 
wherein video data of moving pictures are delineated 10 
into data V1, V2, ... on the time base while audio data 1 
of the sound (for example, the right side sound of the 
stereo sound) are delineated into data A1 1, .... A16, ... 
and audio data 2 of the sound (for example, the left side - 
sound of the stereo sound) are delineated into data is 

A21 A25, .... and other data such as, for example, 

character data to be inserted in a screen are delineated 

into data AUX1 AUX4, ... 

Here, if the data V1, V2, ... of the video data are 
determined as video packets VI, V2, ... respectively 20 
then the video packets V1, V2, ... make a reference for 
construction of packets of the other audio and character 
data. In particular, taking the packet construction for the 
audio data 1 as an example, from among the data A1 1 
to A16 of the video data 1 , the data A1 1 to A13 which 25 
are within a substantially same time as the time of the 
video data V1 are collected to form a first audio 1 packet 
#1 , and the data A14 and A15 which are within a sub- 
stantially same time as the time of the video data V2 are 
collected to form a second audio 1 packet #2. Thus, a so 
stream which includes the audio 1 packet #1 , the audio 
1 packet #2, ... is produced, and the. audio 1 packet #1 
and the audio 1 packet #2 of the stream correspond to a 
PES packet #2 in the pack layer shown in FIG. 4. 

The video packets #1, #2 audio 1 packets #1, 35 

#2 audio 2 packets #1 , #2, ... and character packets 

#1, #2, ... constructed in this manner are time division 
multiplexed such that the first packets (#1) of them are 
time division multiplexed and the second packets (#2) of 
them are time division multiplexed (here, for the conven- 40 
ience of decoding, video data, audio data and character 
data are multiplexed in this order) to construct a pack #1 
and another pack #2, respectively. Thus, a pack stream 
which corresponds to the uppermost layer shown in 
FIG. 4 is formed from the packs #1 , #2, ... 45 

It is to be noted that a packet header for each 
medium of a PES packet includes time stamp PTS code 
information which makes a reference to time information 
which is used to establish synchronism among the 
media upon decoding by the decoding side and so forth, so 

FIG. 3 illustrates an array of picture frames which 
construct one GOP of compression coded pictures in 
one video packet and particularly shows an example 
wherein one GOP is formed from 15 picture frames. In 
particular, one GOP is composed of one I picture frame 55 
M disposed always at the top of the GOP and making a 
base for predictive coding of the other picture frames, 
four P picture frames P1 to P4 formed by forward predic- 
tive coding from the I picture frame and ten B picture 



frames B1 to B10 formed by bidirectional coding from 
the I picture frame and the next P picture frame or from 
two adjacent ones of the P picture frames. Upon decod- 
ing, the pictures are decoded in order of 11 , B1 , B2, P1 , 
.... B10. The picture stream of the video PES packet #1 
in the PES packet layer of FIG. 4 is shown including the 
11, B1, B2. P1 and B3 pictures of FIG. 3, and a picture 
start code which indicates the start of a picture is added 
to the top of each of the picture frames. 

Subsequently, operation of the MPEG coded pic- 
ture decoding apparatus will be described. 

An MPEG coded and multiplexed pack stream of 
the uppermost layer, that is. an MPEG multiplexed bit 
stream, is inputted to the MPEG coded picture decoding 
apparatus via a transmission line such as, for example, 
a high speed digital private line and stored into the 
stream buffer memory 1 via the line interface section 26. 

The buffer memory control section 2 controls the 
stream buffer memory 1 in response to a GOP detection 
signal from the GOP detection section 5 detected from 
a GOP flag of a video PES packet of the PES packet 
layer, and reads out, each time such GOP flag is 
detected, stream data of a pack packet from which the 
GOP has been detected. 

The pack header processing section 3 separates a 
pack header of the pack layer read out from the stream 
buffer memory 1 based on a pack start code and a 
stream ID code in the header, extracts SCR code infor- 
mation for a system clock reference disposed in the 
separated header, and supplies the SCR code informa- 
tion to the clock generation section 15 which produces 
an intra-apparatus clock signal to be used on the decod- 
ing side. 

The PES packet processing section 4 separates a 
PES packet header from the PES packet stream, which 
is inputted from the pack header processing section 3 
and from which the pack header has been separated, 
based on the packet start code and the stream ID code 
in the PES packet header disposed at the top of each 
PES packet, extracts PTS code information, which 
makes a reference for time information, disposed in the 
separated PES packet header, and supplies to the time 
information production section 16 timing information, for 
example, for synchronization between coded video data 
and coded audio data to be used on the decoding side. 
Further, the PES packet processing section 4 separates 
the PES packet stream into PES packets for individual 
media (in FIG. 1 , the construction for separation of char- 
acter PES packets is omitted for convenience of illustra- 
tion), and outputs video PES packets to the video output 
port, outputs audio 1 PES packets and audio 2 PES 
packets to the audio output port and outputs video PES 
packets, from which packet headers have been sepa- 
rated, to the GOP output port. 

Then, the video buffer memory 6 stores the picture 
stream of the video PES packets inputted thereto from 
the video output port of the PES packet processing sec- 
tion 4 while the audio buffer memory 1 9 stores the audio 
stream of the audio PES packets inputted thereto from 



11 



EP 0 798 929 A2 



12 



the audio output port. 

The GOP detection section 5 detects, from a video 
PES packet which has been inputted thereto from the 
GOP output port of the PES packet processing section 
4 and from which the PES packet headers have been 
separated, a GOP flag arranged at the top of the video 
PES packet and outputs and supplies the GOP flag as a 
GOP detection signal to the buffer memory control sec- 
tion 2 in order to allow production of a timing at which 
the individual stream data are to be read out from the 
stream buffer memory 1 and the video buffer memory 6 
after each GOP period. Here, although the first GOP 
flag is not detected unless the first pack packet is read 
out from the stream buffer memory 1 , even if the stream* 
buffer memory 1 is not controlled to read out data there- 
from, since data are discharged from the stream buffer 
memory 1 beginning with data stored first when data 
are stored by more than a predetermined amount, the 
first GOP flag of the first pack stream is detected with 
certainty. 

Consequently, the buffer memory control section 2 
performs reading of the stream buffer memory 1 
described above and simultaneously performs reading 
control of the video buffer memory 6. In particular, while 
data are read out from the video buffer memory 6 in 
units of a video PES packet after each GOP period 
under the control of the buffer memory control section 2, 
those data to be actually outputted to the video MPEG 
decoding section 8 are read out in units of a picture after 
each picture frame period under the control of the buffer 
memory control section 7. 

The video MPEG decoding section 8 MPEG 
decodes a picture stream data read out in units of one 
picture frame and in order of the picture array in one 
GOP of FIG. 8 from the video buffer memory 6 and 
stores the decoded picture stream data into the picture 
decoded data frame memory 90 which is a one frame 
memory (for example, in the NTSC video system of the 
MPEG2, a memory for information of 720 x 480 pixels) 
of the decoded data storage section 9. 

The memory management section 91 of the 
decoded data storage section 9 outputs, after the 
decoded data for one picture frame inputted thereto 
from the video MPEG decoding section 8 are stored 
fully or regularly into the picture decoded data frame 
memory 90, the picture decoded data for the one picture 
frame so that they may be stored into the video display 
frame memory 100 of the video display processing sec- 
tion 10. Here, if decoded data for one frame are not fully 
or regularly stored into the picture decoded data frame 
memory 90, that is, if some picture decoded data 
misses, then the memory management section 91 out- 
puts, together with an error detection signal, error infor- 
mation such as information of a picture position of the 
picture frame, which suffers from the miss of data, 
detected by counting an order number of storages of 
decoded data starting from a time at which I picture 
decoded data were stored together with start code infor- 
mation. On the other hand, when all picture decoded 



data are stored fully or regularly, the memory manage- 
ment section 91 outputs a regular storage signal upon 
completion of the storage and simultaneously outputs 
the stored picture decoded data to the video display 
5 frame memory 100. 

Regular picture decoded data outputted from the 
picture decoded data frame memory 90 are stored for 
several picture frames into the video display frame 
memory 100 of the video display processing section 10. 

w Otherwise, if an error detection signal of a data miss is 
outputted from the memory management section 91 
and a picture re-sending request signal from the error 
information processing section 14 is sent out to the 
MPEG coding multiplexing side via the CPU control 

is section 25 and then, before data are re-sent in response 
to the re-sending request and MPEG decoded and 
stored into the picture decoded data frame memory 90 
again, the picture decoded data for several picture 
frames which have been stored in the video display 

20 frame memory 100 till then are read out so as to be dis- 
played on a screen, then when the next picture frame 
with which the data miss has occurred is displayed on 
the screen, a pause condition is entered while the dis- 
play screen remains displaying a screen of the immedi- 

25 ately preceding picture frame and pictures for several 
picture frames will miss in the pause condition. There- 
fore, in order to assure sufficient time after a re-sending 
request till completion of storage of re-sent picture 
decoded data, picture decoded data for a required 

30 number of picture frames are stored into the video dis- 
play frame memory 1 00. 

Then, the display control section 101 of the video 
display processing section 10 outputs picture decoded 
data read out for each one picture frame from the video 

35 display frame memory 100 so as to be displayed on the 
screen' after each picture frame period under the control 
of the buffer memory control section 7 to the video D/A 
conversion section 12 in response to a video display 
timing signal supplied thereto from the display timing 

40 production section 1 1 . 

Here, the video display timing signal outputted from 
the display timing production section 1 1 is, for example, 
a frame signal of the period of 1/30 second which is a 
video vertical synchronizing signal of the NTSC system. 

45 The frame check detection section 13 supervises 
the output of the memory management section 91 of the 
picture decoded data frame memory 90 after each 
period of the video display timing signal supplied thereto 
from the display timing production section 1 1 , and out- 

so puts nothing if a correct storage signal is received at 
each video displaying timing, but outputs, if an error 
detection signal is received at a video display timing, 
error information to the error information processing 
section 14 together with an error detection signal. 

55 The error information processing section 14 pro- 
duces a video re-sending request signal for requesting 
for re-sending of the video data of the picture frame, 
with which the error has occurred, or for re-sending of 
one GOP incluaing the picture frame to the MPEG cod- 
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ing multiplexing side via the CPU control section 25 
based on the error information including picture position 
information supplied thereto from the memory manage- 
ment section 91 via the frame check detection section 
13, and supplies the video re-sending request signal to 5 
the CPU control section 25. 

The CPU control section 25 controls the video 
MPEG decoding section 8, the audio MPEG decoding 
section 21 and the buffer memory control section 7 via 
the host memory 23 and the host CPU 18 in response 10 
to the intra-apparatus clock signal from the clock gener- 
ation section 15, the intra-apparatus time information 
from the time information production section 16. the 
GOP signal from the GOP detection section 5 and so * 
forth to effect control of MPEG decoding processing, is 
reading processing from the buffer memory and so 
forth. Further, the CPU control section 25 develops a re- 
sending request for video data of the picture frame, with 
which the data miss has occurred, or for video data of 
one GOP including the picture frame to the MPEG cod- 20 
ing multiplexing side via the line interface section 24. 

It is to be noted that the error processing of the error 
information processing section 14 may include not only 
delivery of a video re-sending request for a picture 
frame, with which a data miss has occurred, or for a 25 
GOP including the picture frame but also delivery of a 
request for stopping of sending out of the following 
video data when an error is detected with an I picture or 
a P picture. In particular, if, for example, a miss of data 
occurs on the coding side and this error is not detected 30 
on the coding side and is sent out without being cor- 
rected, then if the miss of data occurs with an I picture 
or a P picture, then there is the possibility that a miss or 
an error of data may occur also with a P picture and B 
pictures which are predictive coded based on the I pic- 35 2. 
ture or with B pictures predictive coded based on the P 
picture. Therefore, in order to prevent an influence of the 
miss or error of data upon a screen display, a request to 
stop sending-out of video data is delivered when an 
error is detected with an I picture or a P picture. 40 

Having now fully described the invention, it will be 
apparent to one of ordinary skill in the art that many 
changes and modifications can be made thereto without 
departing from the spirit and scope of the invention as 
set forth herein. 45 

Claims 

t. An MPEG coded picture decoding apparatus for 
MPEG coded picture data transmitted thereto from so 
an MPEG coding multiplexing side by means of a 
decoding element (8), characterized in that it com- 
prises: 

a memory (90) for temporarily storing video ss 
data MPEG decoded by said decoding element 
(8) in units of one picture frame; 
a memory management section (91) for super- 
vising said memory (90) to detect whether or 



not picture data for one frame are stored into 
said memory (90) completely without a miss; 
a video display processing section (10) includ- 
ing a display memory (100) into which video 
data for one picture frame successively read 
out from said memory (90) are temporarily 
stored for several picture frames and from 
which the video data are read out so as to be 
displayed in the same order in units of one pic- 
ture frame; 

an error processing section (14) for checking 
memory storage information at an output of 
said memory management section (91 ) at each 
video display timing and for outputting, when 
the memory storage information is not output- 
ted from said memory management section 
(91), error information determining that a miss 
of data has occurred with a picture frame 
stored immediately before the display timing 
then and delivering a re-sending request to 
said MPEG coding multiplexing side to re-send 
video data for the one picture frame, with which 
the miss of data has occurred, before the video 
data of the picture frame, with which the miss of 
data has occurred, or a GOP including the pic- 
ture frame are read out from said video display 
processing section (10); and 
a control section (7) for storing video data of 
the one picture frame or the GOP re-sent from 
said MPEG coding multiplexing side into said 
display memory (100) of said video display 
processing section (10) by a time at which the 
re-sent video data are to be displayed. 

An MPEG coded picture decoding apparatus, char- 
acterized in that it comprises: 

a first memory (1) for successively and tempo- 
rarily storing a multiplexed stream inputted 
thereto from an MPEG coding multiplexing side 
via a transmission line and a line input interface 
(26) and multiplexed from MPEG coded bit 
streams of a plurality of media including video 
and audio data; 

a pack header processing section (3) for sepa- 
rating a pack header of the multiplexed stream 
read out from said first memory (1) in response 
to a first control signal and extracting system 
clock reference information for a system clock 
reference; 

a packet processing section (4) for separating a 
packet header from the multiplexed stream 
from said packet header separation section 
from which the pack header has been sepa- 
rated, extracting presentation time stamp infor- 
mation for a time reference and separating the 
multiplexed stream from said pack header 
processing section (3) in units of a packet of 
each of the plurality of media; 
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a GOP detection section (5) for detecting a 
GOP from an MPEG video stream of video 
packets separated by said packet processing 
section (4); 

a first control section (2) for outputting the first s 
control signal and a second control signal in 
response to a GOP detection signal from said 
GOP detection section (5); 
a second memory (6) for temporarily storing 
the MPEG video stream of the video packets w 
separated by said packet processing section 
(4) and being controlled by the second control 
signal from said first control section (2) to read 
out the MPEG video stream temporarily stored 
therein in units of a GOP; is 
a video MPEG decoding section (8) for being 
controlled by a third control signal to MPEG 
decode video data in units of one picture frame 
outputted from said second memory (6); 
a third memory (90) for temporarily storing 20 
decoded video data for one picture frame from 
said video MPEG decoding section (8); 
a memory management section (91) for super- 
vising a storage condition of the decoded video 
data for one picture frame into a storage region 25 
of said third memory (90) and controlling, when 
the storage condition is regular, said third 
memory (90) to output the decoded video data 
for one picture frame stored in said third mem- 
ory (90), but outputting, when the storage con- 30 
dition is irregular, first error information; 
a fourth memory (100) for temporarily storing 
the decoded video data for one picture frame 
outputted from said third memory (90) for a first 
number of picture frames; 35 
a video display control section (101 ) for control- 
ling the decoded video data read out in units of 
one picture frame from said fourth memory 
(100) in response to a fourth control signal so 
that the decoded video data may be outputted 40 
at a video display timing; 
a digital to analog conversion section (12) for 
performing digital to analog conversion of the 
coded video data for each one picture frame 
from said video display control section (101) 45 
and outputting and supplying the resulting ana- 
log decoded video data as an MPEG decoded 
picture signal to an external video viewer; 
a second control section (7) for outputting the 
third and fourth control signals in response to a so 
fifth control signal so that the third control sig- 
nal is supplied to said second memory (6) and 
the fourth control signal is supplied to said 
fourth memory (100); 

a display timing production section (1 1 ) for pro- ss 
ducing and supplying the video display timing 
to said video display control section (101) in 
order to allow the MPEG decoded picture sig- 
nal outputted from said digital to analog con- 



version section (12) to be displayed at a timing 
on a picture monitor of the video viewer; 
an error information processing section (14) for 
supervising an input thereto of the first error 
information outputted from said memory man- 
agement section (91) at a timing of a period 
equal to that of the video display timing output- 
ted from said display timing production section 
(11) and outputting, when the first error infor- 
mation is inputted thereto, a video resending 
request signal based on the error information to 
said MPEG coding multiplexing side to request 
for resending of video data of the same picture 
frame as that of the video data with regard to 
which the first error information is outputted; 
a clock generation section (15) for generating a 
decoding side clock signal to be used for 
MPEG decoding based on the system clock 
reference information extracted by said pack 
header processing section (3); 
a time information production section (16) for 
producing decoding side time information to be 
used for MPEG decoding based on the presen- 
tation time stamp information extracted by said 
packet processing section (4); 
a synchronization timing production section 
(20) for producing an AV synchronization tim- 
ing for synchronization between the video and 
audio data upon MPEG decoding based on the 
decoding side time information outputted from 
said time information production section (16); 
and 

a CPU control section (25) for receiving the 
decoding side clock signal from said clock gen- 
eration section (15), the decoding side time 
information from said time information produc- 
tion section (16), the GOP detection signal 
from said GOP detection section (5) and the 
video re-sending request signal from said error 
information processing section (14) to drive a 
host CPU (18) to control the MPEG decoding 
processing of said MPEG decoding section (8) 
via a CPU bus (1 7) and to output the fifth con- 
trol signal to control said second control section 
(7) and sending out the video re-sending 
request signal to said MPEG coding multiplex- 
ing side via a tine output interface (24) and said 
transmission line. 
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(54) MPEG coded picture decoding apparatus 

(57) An MPEG coded picture decoding apparatus 
includes a stream buffer memory (1) for storing an 
MPEG stream, a pack header processing section (3) for 
separating a pack header of the MPEG stream and 
extracting SCR information, a PES packet processing 
section (4) for separating a header of a PES stream, 
extracting PTS information and separating packets, a 
GOP detection section (5) for detecting a GOP of a 
video stream, a video buffer memory (6) for storing the 
video stream, a video MPEG decoding section (8) for 

FIG. 1 



MPEG decoding the video stream, a decoded data stor- 
age section (9) for storing and managing one picture 
frame, a video display processing section (10) for stor- 
ing several picture frames and reading out at a display 
timing, and an error information processing section (14) 
for supervising a picture frame memory at each display 
timing and developing a video re-sending request when 
the picture frame memory is in an irregular condition. 
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